Tag
12 articles
Sixtyfour's innovative 'Eval Stack' approach challenges traditional AI development by rigorously evaluating agents against expert benchmarks, ensuring accuracy over assumption.
AI leaderboard platform Arena has reached a $100 million valuation after just a year, transitioning from a free tool to a commercial service in September.
Learn about the In the Weights score, a novel AI evaluation metric that analyzes neural network parameters to predict model performance and optimize training.
LifeSciBench is a comprehensive benchmark developed by OpenAI that evaluates AI models on real-life science research tasks, focusing on reasoning and decision-making rather than simple recall. It uses expert-authored rubrics to assess how well AI systems can handle complex scientific workflows.
OpenAI shares a comprehensive playbook to guide third-party evaluations of advanced AI systems, focusing on capabilities, safeguards, and validity.
This explainer explores Anthropic's BioMysteryBench, a new AI evaluation framework designed to test large language models in bioinformatics. It examines how the benchmark works, why it matters for AI development, and what it reveals about AI capabilities in specialized scientific domains.
Galtea raises $3.2M to help enterprises test AI agents, addressing the gap between demo and production performance.
This article explains the trade-offs in AI language model performance, focusing on how models like Grok 4.20 reduce hallucinations but lag behind top-tier models in benchmarks.
A new study by METR reveals that nearly half of AI-generated code that passes industry benchmarks would be rejected by real developers due to quality and maintainability issues.
AI benchmarking startup Arcada Labs is testing five leading AI models as autonomous agents on X, evaluating their real-world social media capabilities.
OpenAI announces it will no longer evaluate SWE-bench Verified due to contamination and data leakage issues. The organization recommends SWE-bench Pro as a replacement.
A new tutorial from MarkTechPost demonstrates how to use TruLens and OpenAI models to build transparent and measurable evaluation pipelines for LLM applications.
